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Abstract 

Private  industry  and  the  Military  both  recognize  the  need  to  develop  mobile  applications 
(apps)  to  meet  the  growing  demand  for  delivering  content  in  a  way  that  supports  end-users’ 
needs  and  preferences.  The  U.S.  Navy  has  been  examining  all  the  conceivable  strategic, 
policy,  and  security  issues  surrounding  mobile  application  development  and  deployment,  but 
limited  Navy  commands  have  had  success  implementing  a  policy  and  development 
methodology  for  meeting  widespread  end-user  needs. 

One  exception  has  been  the  Program  Executive  Office  (PEO)  for  Enterprise  Information 
Systems  (EIS),  a  U.S.  Navy  Program  Executive  Office  whose  mission  is  developing  and 
sustaining  business  Information  Technology  (IT)  systems  for  the  Navy.  One  of  their  primary 
customers,  the  Chief  of  Naval  Personnel,  challenged  PEO  EIS  to  develop  a  strategy  and 
development  methodology  for  quickly  developing  mobile  applications  to  meet  a  variety  of 
Navy  Human  Resource  (HR)  needs. 

PEO  EIS,  through  a  designation  to  one  of  its  Program  Management  Offices  (PMOs) — PMW 
240,  or  the  “Sea  Warrior”  Program — employed  an  innovative  approach  for  design, 
development,  and  acquisition  of  mobile  applications  that  has  allowed  it  to  field  multiple  mobile 
applications  in  just  8-12  weeks  per  application  given  strong  customer  engagement.  To  date, 
PMW  240  has  fielded  eight  applications  in  the  past  year  with  dozens  more  in  the  planning 
and  development  phases. 

This  paper  will  share  the  innovative  methodology.  Systems  Engineering  Technical  Review 
(SETR)  process,  and  Federal  Acquisition  Regulation  (FAR)  insights  that  have  allowed  PMW 
240  to  field  mobile  apps  rapidly.  It  will  also  discuss  some  of  the  challenges  and  next  steps  to 
expanding  the  Navy  HR  mobile  application  capabilities.  Since  PMW  240  Is  an  acquisition 
executor,  all  processes.  Innovations,  and  insights  will  be  presented  from  a  practitioner 
perspective  In  hopes  of  benefiting  other  practitioner  organizations  that  require  mobile 
application  deployment  for  their  end-users. 

Background 

There  has  been  an  unprecedented  level  of  interest  across  the  U.S.  Navy  to  rapidly 
investigate  and  enhance  existing  mobile  technology  capabilities,  primarily  due  to  their 
familiarity,  convenience,  ease  of  use,  and  productivity  benefits.  This  investigation  is 
considering  implementations  that  leverage  Government  Furnished  Equipment  (GFE)  and 
“Bring  Your  Own  Device”  (BYOD)  models. 
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As  the  lead  organization  for  Navy  Enterprise  mobility,  the  Deputy  Chief  of  Naval 
Operations  for  Information  Warfare  held  a  Mobility  Summit  in  October  2014,  which  laid  the 
path  to  develop  a  holistic  view  of  Navy  enterprise  mobility  efforts — supporting  afloat,  ashore, 
and  forward  deployed  operating  environments.  As  a  result,  the  Enterprise  Mobility  Integrated 
Product  Team  (EMIPT)  stood  up  in  January  2015  to  serve  as  the  Navy’s  designated 
advisory  and  action  group  for  all  matters  pertaining  to  Navy  enterprise  mobility  efforts.  The 
team  defined  Enterprise  Mobility  as 

the  suite  of  technologies  and  solutions  that  provides  Navy  personnel  access 
to  information  any  time,  any  place,  and  from  any  device.  Access  may  be 
provided  via  government  and/or  commercial  infrastructure  utilizing  multiple 
device  capabilities,  and  related  network  and  applications  capabilities. 
(Department  of  the  Navy,  2012) 

Notwithstanding  the  demand  for  mobile  application  availability,  there  are  significant 
information  assurance  and  other  technical  and  policy  issues  being  actively  addressed 
across  the  Navy.  Leveraging  existing  guidance.  Navy  Manpower,  Personnel,  Training,  and 
Education  (MPT&E)  leadership  initiated  its  Mobile  Application  Management  effort  using  the 
support  of  the  Program  Executive  Office  for  Enterprise  Information  Systems  (PEO 
EIS)/PMW  240  Sea  Warrior  Program  to  develop  and  deploy  mobile  applications  for  the 
MPT&E  domain.  A  key  element  of  the  PMW  240  tasking  is  to  build  on  government  and 
commercial  best  practices,  document  its  business  and  technical  management  processes, 
and  lay  out  a  path  for  institutionalizing  MPT&E  mobile  application  management  practices. 
This  tasking  is  being  performed  by  the  MPT&E  Mobility  Team,  staffed  by  PMW  240. 

The  PMW  240  Mobility  Team  develops,  oversees  I A  accreditation,  tests,  deploys, 
and  supports  mobile  applications  based  on  requirements  from  the  MPT&E  user  community. 
The  PMW  240  Mobility  Project  allows  for  the  rapid  development  and  deployment  of  mobile 
applications  to  meet  both  end  user  and  Navy  leadership  demand  signals,  with  the  unique 
focus  of  providing  these  applications  on  BYOD  versus  GFE  platforms  and  devices. 

PMW  240’s  specific  involvement  in  mobile  application  development  began  when  the 
Chief  of  Naval  Personnel  (CNP)  challenged  PMW  240  to  build  two  Navy  lieutenants’  mobile 
application  concept  for  Division  Officers.  Six  months  later,  PMW  240  not  only  delivered  the 
eDIVO  (electronic  Division  Officers;  see  Figure  1 )  application,  but  also  the  framework  for  all 
future  MPT&E  mobile  applications.  PMW  240  has  delivered  eight  more  information  and 
training  mobiie  applications  since  eDIVO,  achieving  a  normal  time  to  deliver  from  concept 
approval  in  less  than  four  months,  with  the  development  pipeline  queue  filled  with  mobile 
applications  from  Sailors  and  functional  business  owners. 
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Figure  1.  eDIVO  Guide 


Problem  Statement 

PMW  240  was  tasked  with  quickly  developing  and  delivering  MPT&E  mobile 
applications  to  Sailors  on  their  personal  mobile  devices.  However,  unlike  most  commercial 
mobile  leaders  who  can  define  and  streamline  their  own  procurement  processes,  a  specific 
mobile  application  acquisition  process  did  not  exist  separately  from  the  standard  weapon 
system  acquisition  processes  that  PMW  240  could  leverage.  Using  the  standard  processes 
which  were  developed  for  large-scale  Department  of  Defense  (DoD)  weapon  systems  would 
have  returned  lengthy  development  schedules  and  increased  costs,  which  was 
unacceptable  to  the  Chief  of  Naval  Personnel  and  PEO  EIS. 

Innovative  Solutions  Approach 

To  address  the  challenges  articulated  by  the  problem  statement,  PMW  240 
recognized  it  had  to  acquire  mobile  applications  quickly  and  inexpensively.  To  achieve  this 
goal,  PMW  240  decided  to  use  a  robust  framework  provided  by  an  acquisition  process 
already  tailored  for  IT — the  Abbreviated  Acquisition  Programs  (AAP)  and  Non-Designated 
Program  process.  This  IT  acquisition  process  was  a  necessary  first  step,  but  PMW  240  also 
recognized  that  it  needed  to  “fine  tune”  and  adapt  that  existing  framework  into  one  that  could 
successfully  deliver  lightweight  and  secure  mobile  applications  to  their  end  users  within 
months  of  initial  conception  without  compromising  the  appropriate  quality  control  and 
security  checks  inherent  in  the  current  process.  Figure  2  illustrates  how  PMW  240 
innovatively  tailored  the  standard  weapon  system  acquisition  process  to  address  its  mobile 
application  delivery  challenge. 
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Figure  2.  Robust  DoD  IT  Acquisition  Lifecycle  Process  Tailored  to  the  PMW  240 

Mobile  Application  Process  and  Innovations 

PMW  240  has  now  successfully  implemented  the  acquisition  lifecycle  process 
outlined  in  Figure  2.  The  remainder  of  this  section  will  outline  specific  activities  of  the  Idea, 
Acquisition,  Development,  and  Sustainment  phases  of  this  process,  identify  core  principles 
upon  which  mobile  application  acquisition  is  being  executed,  and  discuss  specific 
acquisition-related  innovations  in  each  of  the  four  phases  of  the  Figure  2  lifecycle  process 
(Cochrane  &  Brown,  2010). 

Mobile  Acquisition  Lifecycle  Overview 

The  High-Level  Operational  Concept  graphic  in  Figure  3  depicts  the  streamlined 
process  the  PMW  240  Mobility  Team  uses  to  identify  mobile  application  requirements,  then 
progressively  lead  those  requirements  through  a  series  of  executable  systems  engineering 
and  project  management  phases  and  decisions  to  a  fully  functioning  and  sustainable  mobile 
application  (PMW  240  Sea  Warrior  Program,  2015b). 
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Figure  3.  PMW  240  Mobile  Application  Development  Process 

The  entire  mobile  application  development  process  flows  from  left  to  right  through 
the  Idea,  Acquisition,  Development,  and  Sustainment  phases,  with  specific  and  important 
activities  involving  both  functional  owners  (customers)  and  PMW  240  in  each  phase. 

The  process  starts  in  the  Idea  phase  with  the  generation  of  ideas  for  new  apps  being 
presented  to  and  evaluated  by  the  Mobile  Application  Group  (MAG),  a  governance  body  that 
prioritizes  mobile  application  development.  MAG  approved  applications  are  assigned  to  the 
PMW  240  Mobility  Team  for  acquisition  (PMW  240  Sea  Warrior  Program,  201 5e). 

The  Acquisition  Phase  starts  when  approved  ideas  are  more  formally  defined 
through  the  generation  of  acquisition  documents  and  data.  Through  PMW  240’s  streamlined 
acquisition  process  and  document  templates,  the  team  is  able  to  rapidly  move  on  these 
application  ideas.  The  PMW  240  team  then  identifies  an  acquisition  strategy,  works  with  the 
application  product  owner  (Navy  subject  matter  expert  organization  who  will  own  the  content 
of  the  application  after  development)  to  create  a  functional  requirements  document  (FRD), 
executes  a  Product  Owner  Agreement  (POA)  with  that  owner,  and  conducts  a  project  kick¬ 
off  with  application  product  owners  to  ensure  their  understanding  and  support  of  the 
application’s  readiness  for  development. 

After  project  kickoff,  PMW  240  enters  the  Development  phase  by  conducting  a  series 
of  tailored  systems  engineering  technical  review  (SETR)  events  that  guide  the  project 
through  requirements  refinement,  design,  development,  test,  and  production  readiness 
decisions  before  the  app  is  deployed  for  use.  Those  SETR  events  will  be  described  in  more 
detail  later  in  this  paper.  Applications  may  be  developed  internally  by  Navy  software 
developers,  by  PMW  240  contracted  software  developers,  or  externally  by  third  party 
developers  who  are  sponsored  by  Navy  MPT&E  functional  leads  or  independent  submitters. 
The  outputs  of  the  Acquisition/Development  Phase  are  a  fielded  mobile  application 
published  on  designated  application  stores. 

In  the  Sustainment  Phase,  the  Product  Owner  provides  updated  content  as  needed 
to  the  developer  who  updates  the  mobile  app  for  publication  via  the  app  store.  Both  the 
Product  Owner  and  the  PMW  240  Mobility  Team  monitor  feedback  on  content,  functionality, 
usability,  and  user  experience  to  determine  upgrades  or  retirement  for  the  app. 

Mobile  Application  Core  Principles 

In  addition  to  the  overarching  acquisition  lifecycle  and  development  process  it 
developed  and  is  following,  PMW  240  recognized  that  it  needed  to  identify  and  follow  some 
core  principles  to  also  guide  its  mobile  application  acquisition  efforts.  These  core  principles 
provide  a  solid  strategic  foundation  on  which  PMW  240  bases  its  mobile  application 
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acquisition  and  ensure  that  certain  performance  and  compliance  requirements  are  met 
(PMW  240  Sea  Warrior  Program,  2015a).  These  principles  are 

•  Simplicity  and  Rapid  Deployment — Mobile  application  projects  should  be 
designed  with  rapid  deployment  and  simplicity  in  mind,  wherever  possible. 

•  Performance — The  application  must  follow  standard  iOS  and  Android 
development  practices  to  ensure  a  normal  level  of  memory  consumption. 

•  Security — The  application  must  adhere  to  requirements  and  specifications 
outlined  in  the  Cybersecurity  Mobiie  Appiication  Checklist,  Fortify  scans,  and 
their  respective  references. 

•  Compiiance — The  application  must  compiy  with  standard  mobile  platform 
vendor  development  guidelines  outlined  in  official  licensing  and  distribution 
agreements. 

•  User  Documentation  and  Training — The  application  should  make  all 
documentation,  iifecycle  management,  and  training  information  pubiicaiiy 
avaiiabie  as  needed.  Application  tutoriais  are  a  preferred  method  for  training 
users  of  mobile  applications  how  to  perform  necessary  functions  to  utilize  the 
application  effectively. 

•  Maintenance/Sustainment — Mobile  application  projects  should  be  designed 
to  reduce  the  burden  of  maintenance  and  other  sustainment  actions.  Before 
using  any  feature  or  supporting  software,  a  developer  must  first  search  for 
reports  indicating  software  and  support  will  not  sunset  in  the  near  future. 

Also,  the  developer  must  compare  alternatives  with  respect  to  proven 
software  and  support  longevity,  and  reputation  for  ease  of  maintenance. 

•  Feedback — The  user  must  have  the  capability  to  email  feedback  directly  to 
the  NAVY  31 1  helpdesk.  In  addition,  mobile  application  projects  will  use  a 
Commercial  Off  the  Shelf  (COTS)  capability  to  capture  feedback  within  the 
application  and  subsequently  collate,  tag,  and  send  that  data  to  NAVY  31 1 
when  appropriate. 

o  Each  application  will  be  issued  its  own  email  address  to  facilitate 
communication  between  the  COTS  software  and  NAVY  31 1 . 

o  The  COTS  software  also  provides  the  capability  to  capture  feedback 
from  various  App  Stores,  Facebook,  Twitter,  and  other  social  media 
resources,  creating  tickets  from  the  feedback  it  discovers. 

Idea  Phase  Innovations 

Although  the  Idea  phase  of  the  lifecycle  is  relatively  short  and  simple,  PMW  240  has 
applied  innovative  guidance  and  tools  to  both  accelerate  and  simplify  this  phase. 

Streamlined  Technical  and  Programmatic  Documentation 

Agile  Mobility  Plan 

PMW  240  deveioped  the  Agile  Mobility  Plan  (AMP)  to  provide  technical  information 
related  to  the  development,  cybersecurity,  testing,  deployment,  and  sustainment  of  PMW 
240  mobile  applications.  The  information  contained  in  the  document  represents  what  is 
common  to  aii  PMW  240  mobiie  appiication  investments.  The  AMP  is  the  biueprint  for  the 
technical  conduct  and  control  of  PMW  240  mobile  applications  from  inception  through 
sustainment.  As  a  lightweight,  tailored  version  of  the  PMW  240  Systems  Engineering  Plan 
(SEP),  this  25-page  document  (innovativeiy  short  and  concise)  highlights  only  the  aspects  of 
systems  engineering  that  are  prevalent  to  the  mobile  application  lifecycle.  This  aliows  for  a 
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purposeful,  pointed  document  resulting  in  the  rapid  development  and  deployment  of  mobile 
applications  to  meet  both  end  user  and  Navy  leadership  demand  signals  (PMW  240  Sea 
Warrior  Program,  2015b). 

Agile  Mobility  Management  Plan 

PMW  240  also  developed  the  Agile  Mobility  Management  Plan  as  a  companion 
document  to  the  AMP.  The  Agile  Mobility  Management  Plan  provides  for  management  and 
governance  of  PMW  240  mobile  application  investments.  The  document  specifies  and 
delegates  the  Mobility  Project  Decision  Authority  (MDA)  from  the  PMW  240  Program 
Manager  down  to  a  lower  level,  the  Principal  Assistant  Program  Manager  (PAPM),  for 
expedited  decisions  and  more  availability.  Through  delegation  of  this  authority  within  the 
program  office,  project  milestones,  management  policies,  and  governance  decisions  occur 
at  a  more  rapid  pace,  permitting  the  PMW  240  team  to  deliver  more  applications  in  less  time 
(PMW  240  Sea  Warrior  Program,  2015a). 

Mobile  Adjudication  Board 

PMW  240  established  the  Mobility  Adjudication  Board  (MAB)  to  manage 
requirements,  defect  resolution,  and  other  mobile  application  project  issues  as  required;  it  is 
also  a  forum  used  to  implement  the  fundamental  change  management  process  of 
configuration  control  during  planning,  development,  deployment,  and  sustainment.  The 
Mobility  Adjudication  Board  Charter  (MABC)  enables  a  disciplined  approach  and  visibility  for 
the  approval,  disapproval,  and  prioritization  of  new  or  existing  requirements.  It  is  a  critical 
component  to  maintaining  the  known  configuration  and  ensuring  all  changes  are  approved 
prior  to  implementation. 

The  MABC  has  the  Scope  of  Authority  (SoA)  for  mobile  app  changes  to  the 
configuration  baseline  during  the  planning,  development,  deployment,  and  sustainment 
phases  of  the  program.  Mobility  projects  normally  progress  at  a  higher  rate  of  speed  than 
standard  web  application  endeavors.  For  this  reason,  a  single,  very  lightweight  governing 
structure  handles  configuration  management  oversight  during  development  (normally,  more 
cumbersome  Program  Review  Board  [PRB]  or  Configuration  Control  Board  [CCB]  structures 
in  full  IT  system  acquisitions).  This  innovatively  lightweight  structure  ensures  issues  are 
handled  in  a  timely  manner  by  the  appropriate  oversight  and  combines  two  traditional 
processes — PRB  and  CCB — into  a  single  efficient  review  team  (PMW  240  Sea  Warrior 
Program,  201 5d). 

Multiple  Entry  Points 

PMW  240  receives  mobile  applications  that  fall  under  various  states  of  maturity 
within  the  lifecycle  and  allows  for  all  to  enter  into  its  lifecycle  process.  Most  applications  are 
proposed  in  the  form  of  less  mature  ideas,  but  some  are  maturing  and  already  in  some 
phase  of  a  development  state,  or  are  fully  built  and  ready  to  be  published  into  the  application 
store.  To  conserve  resources  and  recognize  the  lifecycle  maturity  of  these  various 
applications,  PMW  240  considers  the  current  state  of  the  application  to  determine  where 
and  how  to  categorize  it.  This  allows  for  a  customized  yet  expedited  entry  into  the 
application  store  while  verifying  the  application  meets  the  proper  exit  criteria  for  deployment. 

Idea  Mailbox 

PMW  240  acquires  application  ideas  through  a  variety  of  sources,  including 
leadership  direction,  command  interest,  and  line  of  business  owner  ideas.  PMW  240  also 
utilizes  a  digital  mailbox  advertised  on  Navy  media.  This  innovative  mailbox,  seen  in  Figure 
4,  navyapps@navy.mil,  receives  ideas  for  new  applications  from  both  civilians  and  Sailors 
and  is  checked  on  a  weekly  basis  to  ensure  new  and  fresh  ideas  for  applications  from 
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practitioners  are  in  the  forefront  of  application  development  consideration.  This  mailbox  is  an 
innovative  approach  to  soliciting  mobile  application  ideas  directly  from  the  end-users  who 
will  benefit  from  them. 


Got  an 
app 
idea? 


Solve  a  problem,  save  time  or 
improve  a  process 


NavyApps@navy.mil 


We  might  Just  make  an  app  for  that! 


Figure  4.  PMW  240’s  Application  Mailbox 

Mobile  Action  Group  Review  and  Approval 

On  a  quarterly  basis,  PMW  240  briefs  the  collected  application  ideas  to  the  Mobile 
Application  Group  (MAG)  for  investment  approval.  If  an  application  is  not  chosen  for 
immediate  investment,  an  informational  quad  chart  detailing  salient  information  for  each 
application  idea  is  entered  into  the  Mobility  Team’s  application  backlog  and  will  be 
reconsidered  by  the  MAG  at  the  following  quarterly  meeting.  If  the  MAG  approves  an 
application  idea,  the  PMW  240  Mobility  Team  performs  required  contracting  actions  which 
signal  the  beginning  of  the  acquisition  phase.  Using  a  quarterly  time-driven  review  period 
keeps  the  investments  current  and  provides  PMW  240  with  regular  direction  on  applications 
to  best  align  with  the  leadership  and  end-user  interest.  Such  frequent  review  of 
requirements  and  prioritization  is  innovative  for  IT  acquisition,  to  say  nothing  of  standard 
DoD  5000  weapon  systems  prescribed  processes  (PMW  240  Sea  Warrior  Program,  201 5e). 

Acquisition  Phase  Innovations 

The  acquisition  phase  of  anything  the  DoD  procures  is  not  generally  considered  to 
be  a  space  where  innovation  can  flourish.  However,  PMW  240  has  implemented  innovative 
approaches  to  allow  agility  in  acquiring  new  mobile  applications. 

FAR/Contracting 

Per  the  Federal  Acquisition  Regulation  (FAR),  contract  negotiation  and  execution 
can  require  extensive  effort  and  wait  time  for  an  initial  contract  award  and  additional  time  for 
follow-on  task  order  awards  against  an  Indefinite  Delivery/Indefinite  Quantity  (ID/IQ)  or 
Multiple  Award  Contract  (MAC)  vehicle.  To  support  rapid  mobile  application  development, 
PMW  240  quickly  recognized  it  needed  a  very  flexible  and  responsive  contracting  strategy 
and  associated  vehicle  that  FAR-prescribed  competitive  or  negotiated  contracting 
procedures  might  not  allow. 

As  a  result  of  the  unique  MNP  mobile  app  development  needs,  PMW  240  conducted 
market  research  to  identify  industry  standard  timelines  and  costs  for  developing  the  types  of 
mobile  apps  under  consideration.  Based  on  that  data,  PMW  240  alpha-negotiated  an  ID/IQ 
contract  vehicle  with  an  economically-disadvantaged  woman-owned  small  business.  That 
vehicle  contained  Firm-Fixed  Price  (FFP)  Contract  Line  Item  Numbers  (CLINs)  for  small, 
medium,  and  large  application  development,  as  well  as  for  maintenance  tasking  on  an  app 
by  app  basis.  The  CLIN  values  were  based  on  the  industry  standard  costs  and  allow  PMW 
240  to  award  new  task  orders  (TOs)  selecting  the  needed  CLINs  In  a  matter  of  days.  That 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  EOR  INFORMED  CHANGE 


-209- 


TO  award  speed  significantly  decreases  the  overall  acquisition  phase  time  requirements 
(PMW  240  Sea  Warrior  Program,  2015a). 

Product  Owner  Agreement 

Prior  to  beginning  any  development,  the  Mobility  Team  works  with  the  Product 
Owners  to  negotiate  the  Product  Owner  Agreement  (POA).  The  POA  is  a  lightweight 
document  comparable  to  a  larger  program’s  Memorandum  of  Agreement  (MOA)  that 
explains  the  responsibilities  of  the  Product  Owner  and  PMW  240  throughout  the  application 
lifecycle  and  ensures  that  an  application  is  maintained  following  publication  into  the 
application  stores.  The  POA  explains  that  the  Product  Owner  is  responsible  for  any  content- 
related  changes,  including  notifying  the  PMW  240  team  of  any  policy,  link,  or  material 
updates,  and  the  PMW  240  team  is  required  to  handle  any  technical  changes,  such  as  bug 
fixes  and  operating  system  updates.  This  document  is  negotiated  and  signed  by  the  two 
participating  teams  and  reposed  under  configuration  control.  The  lightweight  nature  of  the 
POA  is  innovative  in  that  it  allows  signature  at  a  lower  organizational  level,  so  it  requires 
less  oversight,  allowing  the  development  on  the  application  to  begin  sooner  (PMW  240  Sea 
Warrior  Program,  2016a). 

FRD  Templates/Flexible  Requirements  Gathering 

In  addition  to  the  POA,  PMW  240  requires  a  Functional  Requirements  Document 
(FRD)  to  be  completed  and  signed  before  beginning  development.  The  Mobility  Team  has 
three  FRD  templates  depending  on  the  type  of  the  application  to  be  built:  an  aggregated 
content  application,  a  training  application,  and  a  hybrid  application  (content  and  training). 
The  most  fitting  template  is  then  customized  through  a  series  of  rapid  meetings  with  the 
PMW  240  team  and  the  Product  Owners  and  reviewed  by  the  development  team  for  any 
needed  clarification.  Once  all  parties  are  confident  that  the  FRD  captures  the  vision  for  the 
application,  the  document  is  signed  out  and  the  development  phase  can  begin  with  the 
Tailored  Mobile  Design  Review  (TMDR).  This  innovatively  rapid  requirements  gathering  and 
clarification  process  using  these  pre-defined  templates  generally  requires  no  more  than  10 
business  days,  which  is  an  extremely  short  timeline  compared  to  standard  IT  and  weapon 
systems  acquisitions  timelines  for  similar  activities  (PMW  240  Sea  Warrior  Program,  2015c). 

Deveiopment  Phase  Innovations 

The  Development  phase  of  the  lifecycle  is  usually  the  longest  phase  of  any 
acquisition,  and  it  is  for  PMW  240  mobile  applications  as  well.  To  decrease  required 
development  time  as  much  as  possible,  PMW  240  has  implemented  innovative  guidance 
and  oversight. 

“Official”  Developer  Account 

With  a  diverse  set  of  product  owners  working  with  PMW  240  to  develop  the 
applications,  it  is  important  to  adhere  to  specific  standards  and  meet  certain  thresholds 
when  it  is  time  for  production.  PMW  240  established  developer  accounts  for  the  public 
application  stores  (Apple  and  Google)  and  is  the  clearing  house  for  all  official  Navy  MPT&E 
mobile  applications.  The  streamlined  process  for  publication  and  production  ensures  each 
application  meets  the  exit  criteria  for  the  PMW  240  process  and  the  entrance  criteria  for 
these  public  application  stores.  The  efficiency  of  this  singular  clearing  house  point  provides 
a  structured  process  and  effective  release  methodology. 

Tailored  Technical  Events 

The  PMW  240  Technical  Event  Process  (TEP)  guidebook  provides  guidance  for 
planning  Systems  Engineering  Technical  Reviews  (SETR)  events.  However,  that  document 
generally  guides  development  through  a  waterfall  approach  that  requires  months  and  even 
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years  of  technical  events  to  deliver  working  software.  PMW  240  innovatively  tailored  the 
SETR  guidance  to  match  the  agile  methodology  it  has  implemented  to  deliver  MPT&E 
mobile  apps  in  weeks  and  months  instead  of  years.  The  following  are  the  critical  tailored 
technical  events  required  to  track  progress  for  each  mobile  application.  Each  of  the  four 
technical  events  listed  below  are  one  hour  in  length  and  require  participation  from  the 
Product  Owners,  PMW  240,  the  development  team,  and  representatives  from  Cybersecurity, 
the  Public  Affairs  Office  (PAO),  Enterprise  Change  Management  (ECM),  Configuration 
Management  (CM),  Test,  and  Logistics. 

•  Kickoff  Meeting  (KO) — After  a  mobile  app  project  receives  MAG  approval,  a 
Kickoff  meeting  is  held  to  introduce  team  members  from  different 
competencies  and  stakeholders  to  establish  the  expectations  for 
development/deliver,  general  procedures  to  be  followed,  priorities,  schedule, 
and  clear  assignment  of  roles  and  responsibilities. 

•  Tailored  Mobile  Design  Review  (TMDR) — The  TMDR  is  a  tailored 
combination  of  three  standard  technical  reviews:  System  Requirements 
Review  (SRR),  System  Functional  Review  (SFR),  and  Preliminary  Design 
Review  (PDR).  Conducted  by  the  Mobility  Assistant  Project  Manager- 
Engineer  (APM-E),  this  review  ensures  the  preliminary  design  of  the 
application  meets  all  functional  requirements  and  the  initial  and  allocated 
baselines  for  development,  test,  and  deployment  have  been  established. 
Combining  these  events  allows  PMW  240  to  shorten  the  data  collection  and 
review  timeline  to  only  pertinent  information. 

•  Test  Readiness  Review  (TRR) — The  Mobility  APM-E  conducts  this  review 
once  the  application’s  initial  development  effort  has  been  completed.  This 
review  will  assess  the  application’s  readiness  to  begin  initial  formal  testing 
procedures.  These  procedures  include  testing  the  application  on  PMW  240- 
owned  mobile  devices  (including  both  smartphones  and  tablets),  as  well  as 
on  mobile  platform  simulator  software.  It  also  includes  conducting  needed 
security  scans. 

•  Production  Readiness  Review  (PRR) — The  Mobility  APM-E  and  Mobility 
PD  conduct  the  Production  Readiness  Review  (PRR)  following  the 
completion  of  initial  testing  on  PMW  240-owned  smartphones  and  tablets. 
This  review  will  analyze  the  application’s  readiness  to  begin  the  migration 
process  to  the  target  mobile  application  stores.  This  analysis  will  include 
further  testing  of  the  application  on  personally-owned  devices  to  ensure  the 
integrity  of  its  performance. 

The  PMW  240  Mobility  Project  Team  works  with  the  mobile  application  developer  to 
ensure  the  evaluation  criteria  for  entrance  and  exit  of  particular  technical  events  are 
appropriate  to  the  level  of  effort,  cost,  schedule,  and  complexity  of  the  mobile  application. 

The  240  mobility  project  team  reviews  developer-crafted  test  cases  to  ensure  they 
prove  completion  of  capabilities  they  are  written  to  test.  The  contractor  performs  a  final 
quality  assurance  testing  phase  after  the  final  development  iteration.  After  this  final  testing 
iteration,  the  government  enters  the  final  acceptance  testing  procedures.  The  PMW  240 
Mobility  Project  Team  also  integrates  security  and  usability  testing  and  evaluation  into  the 
development  process  to  streamline  testing  cycles  and  overall  impact  to  cost  and  schedule. 
The  developer  ensures  that  prior  to  each  iteration  release,  the  developed  application  has 
gone  through  a  cycle  of  developer  level  testing  to  ensure  functionality  intended  for 
demonstration  and  preview  is  functioning  as  expected. 
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During  the  government’s  acceptance  testing,  the  government  solicits  the  necessary 
testing  resources  that  align  with  the  application’s  target  audience,  and  the  Test  Team  lead 
verifies  all  application  functionality  works  as  designed. 

Final  application  approval  and  permission  to  release  the  application  is  at  the 
discretion  of  the  PMW  240  Mobility  Team  Decision  Authority  after  the  Development  Team 
adjudicates  and  addresses  submitted  defects  and  comments. 

Compared  to  normal  weapon  system  and  IT  SETR  events  and  timelines,  this 
process  tailored  for  PMW  240  mobile  applications  is  highly  innovative  and  extremely  fast  in 
delivering  capability  to  end-users  (PMW  240  Sea  Warrior  Program,  2015b). 

MPT&E  Mobile  Application  Toolbox 

In  an  effort  to  encourage  application  proliferation  for  the  Navy  MPT&E  community, 
PMW  240  developed  a  “toolbox”  that  is  accessible  to  civilian,  active  duty,  and  reserve 
members  of  the  Navy.  While  PMW  240  would  still  be  the  clearing  house  and  publishing 
authority,  the  toolkit  provides  tips,  style  guidance,  information  on  development 
environments,  and  tricks  for  building  specific  application  components  for  any  development 
audience  that  wishes  to  build  mobile  applications  that  look,  feel,  function,  and  are 
compatible  with  those  PMW  240  has  built  for  the  Navy  MPT&E  community.  The  toolbox  is 
designed  as  a  self-sustaining  wiki,  meaning  that  developers  can  use  the  site  to  post 
questions,  read  topic  forums,  and  even  contact  the  PMW  240  mobility  team  for  specific 
questions.  The  website  will  highlight  sample  projects  and  serve  as  an  additional  execution 
arm  to  the  work  being  completed  in  PMW  240.  The  use  of  a  toolkit,  shown  in  Figure  5, 
allows  developers  to  structure  their  application  to  appear  and  operate  as  an  official  U.S. 
Navy  application,  yet  encourages  development  by  third  parties.  It  is  an  extremely  innovative 
and  collaborative  approach  that  allows  any  capable  entity  to  develop  MPT&E  approved 
mobile  applications,  instead  of  restricting  that  ability  to  one  single  vendor  or  in-house 
Government  development  team  (PMW  240  Sea  Warrior  Program,  2016b). 


Figure  5.  MPT&E  Mobility  Toolkit 
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Sustainment  Phase  Innovations 

Although  the  Idea  phase  of  the  lifecycle  is  relatively  short  and  simple,  PMW  240  has 
applied  innovative  guidance  and  tools  to  both  accelerate  and  simplify  this  phase. 

Customized  Sustainment  Plans  and  Reviews 

Once  an  application  has  been  released  for  use,  PMW  240’s  primary  task  is  to  ensure 
the  content  and  technology  is  current,  functional,  and  accessible  to  the  user  base.  As 
discussed  previously,  each  Product  Owner  signs  a  POA  before  development  begins,  and 
the  POA  is  followed  throughout  sustainment  of  the  application.  On  a  quarterly  basis,  the 
application  is  reviewed  to  assess  the  value  of  the  investment.  Along  with  metrics  and 
feedback,  discussed  below,  the  internal  PMW  240  team  reviews  the  need  for  any  updates 
(either  content-related  or  technical)  to  determine  whether  an  application’s  state  is 
acceptable.  On  a  biannual  basis,  the  Product  Owners  are  invited  to  the  reviews  and  discuss 
the  feasibility  of  an  upgrade,  application  usage,  alignment  with  the  Product  Owner’s  team, 
and  to  decide  whether  the  application’s  status  warrants  continued  sustainment  funding. 
Monitoring  and  evaluating  the  applications  every  three  months  prevents  stagnant  content, 
unwarranted  investment,  and  insightful  trend  analysis,  and  fosters  the  relationship  with  the 
Product  Owners — all  on  an  innovatively  manageable  level  and  with  a  minimal  time 
investment. 

Metric  and  Feedback  Collection 

PMW  240  collects  metrics  and  feedback  from  each  of  the  applications  in  order  to 
better  assess  the  status  of  any  particular  application  and  use  the  results  to  determine 
continued  investment.  Metrics  are  aggregated  from  the  application  stores  and  a  built-in 
feedback  mechanism  within  the  application.  From  the  public  stores,  PMW  240  can  see  star 
ratings,  comments,  number  of  downloads  per  day,  and  devices  that  use  the  application. 

From  the  inherent  mobile  applications  feedback  mechanism,  PMW  240  can  view,  respond 
to,  and  route  comments  to  the  development  or  product  owner  team  for  consideration. 
Comments  are  often  in  the  form  of  suggestions  for  additional  content/functionality  or  reports 
of  bugs.  The  above-mentioned  feedback  is  collected  on  a  weekly  basis,  and  the  compiled 
version  is  distributed  on  a  monthly  basis  for  review.  The  PMW  240  team  can  actively  monitor 
an  application’s  usage,  end  user  reactions,  and  any  issues  and  incorporate  any  resulting 
changes  into  future  builds  of  the  application.  This  innovatively  thorough  yet  rapid  and  easy- 
to-decipher  data  collection  and  monitoring  allows  for  a  direct  feedback  loop  and  response 
adjudication  to  better  serve  the  end  user’s  needs. 

Challenges  and  Next  Steps 

Mobility  within  the  Navy  will  continue  to  grow  and  reach  a  broader  audience.  With 
this  growth,  the  demand  for  more  mobile  capabilities,  including  transactional  applications 
that  interact  with  current  DoD  systems,  will  increase.  There  are,  however,  a  number  of 
challenges  facing  the  Navy — particularly  in  the  use  case  of  BYOD  mobile  platforms — to 
ensure  its  workforce  can  fully  utilize  mobile  capabilities  for  all  their  mission  requirements. 
Some  of  these  challenges  include  using  Derived  Credentials,  opening  an  Official  Navy  App 
Store,  and  implementing  a  Mobile  Application  Management  (MAM)  framework. 

Derived  Credentials 

Supporting  secure  access  to  mobile  devices  through  ‘Derived  Credentials’  (a 
National  Institute  of  Science  and  Technology  coined  term  to  describe  cryptographic 
credentials  derived  from  Personal  Identity  Verification  [PIV]  and  Common  Access  Cards 
[CACs])  is  one  of  the  Navy’s  and  U.S.  Government’s  biggest  challenges  for  enabling  its 
mobile  workforce  to  securely  access  and  authenticate  mobile  devices  interacting  with 
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Government  data.  The  current  use  of  physical  CACs  and  card  readers  limits  the  use  cases 
of  usable  mobile  devices  and  is  un-scalable,  resulting  in  high  costs  to  implementation.  Using 
software,  micro-hardware,  or  other  cryptographic  methods  of  access  and  authentication  will 
have  to  be  developed,  tested,  and  put  into  production  before  the  Navy  fully  realizes  the  full 
suite  of  mobile  capabilities  currently  available  to  the  commercial  world. 

Navy  App  Store 

Providing  Sailors  and  civilians  access  to  a  full  suite  of  official  Navy  mobile 
applications,  designed  to  enable  their  day-to-day  work,  will  ensure  they  have  access  to 
officially  authorized  Navy  information  and  applications.  If  and  when  realized,  this  “Navy  App 
Store”  could  serve  both  GFE  and  BYOD  platforms/devices.  Establishing  this  app  store  will 
provide  a  single  location  for  Sailors  and  civilians  to  access  Navy  applications  and  content 
without  fear  of  downloading  a  fake  or  malicious  Navy  application  in  the  open  commercial 
app  stores. 

MAM  Framework 

Along  with  an  official  Navy  application  store,  utilizing  a  MAM  service  to  manage  the 
growing  number  of  Navy  applications  will  be  vital  to  sustainment.  Keeping  applications  up  to 
date  with  their  respective  operating  system  and  hardware  platforms,  as  well  as  content 
updates,  will  ensure  the  applications  end-users  will  have  fully  operational  apps  with  current 
information.  A  MAM  can  also  provide  robust  security  for  Navy  applications  when  loaded  to  a 
Sailor’s  personal  device  which  may  allow  for  transactional  applications  that  connect  securely 
with  DoD  systems  while  restricting  access  to  any  personal  data  on  the  device.  PMW  240  is 
currently  performing  a  Material  Solutions  Analysis  (MSA)  on  various  MAM  vendors  and  will 
assess  potential  application  use  cases  for  future  production. 

Conclusion 

PMW  240  has  developed  a  streamlined  and  agile  process  to  securely  acquire  and 
deliver  high  quality  mobile  MPT&E  applications  to  Sailors  and  civilians.  As  the  appetite  for 
mobile  applications  and  information  consumption  continues  to  grow,  PMW  240  will  continue 
to  be  flexible  and  scalable  with  its  acquisition  and  associated  mobile  application  fielding 
processes  to  meet  end  user  and  Department  of  the  Navy  future  needs  while  maintaining 
information  security  and  assurance  standards. 
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Problem  Statement 


■  A  specific  Mobile  Application  acquisition  process  does 
not  exist  separateiy  from  the  standard  Platform/Weapon 
System  acquisition  process 

*  PMW  240  needed  rapid  design,  development,  test,  and 
depioyment  process  due  to  the  nature  &  requirements  of  mobile 
apps 


Background 

PMW  240 

■  Interest  across  the  United  States  Navy  to  rapidly 
investigate  and  enhance  existing  mobiie  technoiogy 

•  Familiarity,  convenience,  ease  of  use,  and  productivity  benefits 

■  Deputy  Chief  of  Navai  Operations  for  information 
Warfare  heid  a  Mobiiity  Summit  in  October  2014 

«  Laid  groundwork  to  develop  a  holistic  view  of  Navy  enterprise 
mobility  development  efforts 

■  Enterprise  Mobility  integrated  Product  Team  (EMiPT) 
estabiished  to  pursue  common  mobility  vision  across 
Navy 
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Innovative  Sointions  Annroach 


r.  *  LJtm  Cyol* 


Robust  DoD  IT  Acquisition  Lifecycle  process 
tailored  to  the  PMW  240  Mobile  Application  Process 
with  Innovations  in  each  phase 


Streamlined 
Technical  & 
Programmatic 
Documentation 
Mobile  Adjudication 
Board 

Multiple  Entry 
Points 

Idea  Mailbox 
MAG  vetting 


FAR/Contracting 

Product  Owner 

Agreement 

FRD  templates/ 

Flexible 

Requirements 

Gathering 


“Official”  Developer 
Account 

Tailored  Technical 
Events 
MPTE  Mobile 
Application  Toolbox 


Customized 
Sustainment  Plans 
and  Reviews 
Metric  and 
Feedback  Collection 
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Challenges  and  Next  Stens 


■  Challenges 

■  DoD  5000  processes  &  policies  for  rapid 
IT  &  Mobile  Applications  acquisition 

■  Centralized  Navy  Mobile  Strategy 

■  Cybersecurity  Support 

■  Structured  Policies  and  Procedures 


•  Next  Steps 

■  Mobile  Application/Device  Security 

■  Mobile  Application  Management  (MAM) 

■  Mobile  Content  Management  (MCM) 

■  Navy  App  Store 


..III  PHONE  e 


4:19  PM 


eDIVO 


OPSEG  Domestic 


Sea  Warrior 


PROGRAM 

PMW  240 


QuestionsP 


Got  an 

Solve  a  problem,  save  time  or 
improve  a  process 

app 

NavyApps@navy.mil 

idea? 

We  might  just  make  an  app  for  that! 
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